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1 Introduction 



Geometric approaches to cognition in general and to symbolic computation in par- 
ticular became increasingly popular during the last two decades. They comprise con- 
ceptual spaces for sensory representations (Gardenforsl 2004]). latent sem antic analy- 
sis for the meanings of nouns and verbs (Cederberg and Widdows 2003), and tensor 

product representations for compositional semantics (Blutner 20 09..Aerts..2009i). Ac- 

cordin g to the dynamical system approach to cognition ( van Gel deJl998l beim Graben and Potthasl 
mental states and their temporal evolution are represented as states and trajec- 



tories in a dynamical sys tem's state space. This ap proach has been used, e.g., for mod- 
eling logic al inferences jBalkenius a nd Gardenfo rsll991 , Mizraiill992 ) and language 
processes ( beim Graben et aJ 2008at ITabori 12009 ). Interpreting the states of a dy- 
namical system as activation vectors of neural networks, inc ludes also connectionist 
appro aches o f cognitive modeUng into georn etric cognition dGerth and beim Graben 
2009[lHuvckll2009 . IVosse and Kempenl2009h . 



One particularly significant contribution in th is direction is Smolensky's Inte- 

grated Connectionist/Symbolic Architecture (ICS) (Smolensky 2006, Smolens ky and Legendre 
|2006a). This is a dual-aspect approach where subsymbolic dynamics of neural acti- 
vation patterns at a lower-level description become interpreted as symbolic cognitive 
computations at a higher-level description by means of filler/role bindings through 
tensor p roduct representations. Closely related to ICS is dynamic cognitive modeling 
(DCM) ( beim Graben and PotthastI 20091 : in pressi) . which is a top-down approach for 
the construction of neurodynamical systems from symbolic representations in con- 
tinuous time. 



So far, ICS/DCM architectures have been successfully employed for phonolog- 
ical ([Sm olensky 2006, Smolensky and Legendre 2006a) and syntactic computations 
(ISmolenskvll2006i [Smolensky and Legendrell2006al Ibeim Graben et alll2008ah in the 



fields of computational linguistics and computational psycholinguistics using mainly 
conte xt-free grammars and appropriate push-down automata (Hopcroft and Ullman] 
Il979i) . However, as natural languages are known to belong to the co mplexity class 



of mildly context-sensitive languages within the Chomsky hierarchy (lShiebeiill985 



[SLabler 2004), more sophisticated formal grammars have been developed, including 
tree-adjoining g rammars (TAG) djoshi et all 1 19751). mult i ple context-free grammar s 
(ISeki et allll991l) and minimalist grammars (' Stableil[l997[ (stabler and Keenanll2003b . 
In particular, Stabler's formalism of minimalist grammars (MG) codifies most con 
cepts of generative li nguistics (e.g. from Government and Bi nding Theory (Chomsk 



1981 



Haegemanll99 4) and Chomsky's Minimalist Program (^Choms kvl 1 995i„Weinber! 



200 Ih ^ in a mathematically rigorous manner. In early MG this has been achieved by 
defining minimalist trees and the necessary transformations by means of set and graph 
theoretic operations. In its later development, minima list trees have been aba ndoned 
in favor of chain-based calculus (Harkema 2001, Sta bler and Keenanll2003b due to 
Harkema's statement that "the geometry of a [minimalist] tree is a derivational arti- 
fact of no relevance [...] " (lHarkemal200ll p. 82). Based on these results MG could be 
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recast into mul tiple context-free grammars for further investigation (lMichaelisll200ll 
Harkemall200lb . 



Recently, iGerthI (l2006h and iGerth and beim Graben (l2009h revived the ideas of 



early MG, by presenting two ICS/DCM studies for the processing of minimalist 
grammars in geometric representation spaces, because minimalist tree representa- 
tions could be of significance for psycholinguistic processing performance. In these 
studies, different filler/role bindings for minimalist feature arrays and minimalist trees 
have been used: o ne purely arithmetic representation for filler features and syntactic 
roles (lGerthll2006l) : and another one, combining a rithmetic and numerical r epresenta- 



tions into a fractal tensor product representation ( Gerth and beim Grabenl2"009.) . Until 
now, these studies lack proper theoretical justification by means of rigorous mathe- 
matical treatment. The present work aims at delivering the required proofs. More- 
over, based on the metric properties of representation space, we present an extension 
of MG toward harmonic MG, for providing a complexity measure of minimalist trees 
that might be of relevance for psycholinguistics. 

The paper is structured as follows. In Sec. |2] we algebraically recapitulate Sta- 
bler's original proposals for minimalist grammars which is required for subsequent 
dynamic cognitive modeling. We also illustrate the abstract theory by means of a par- 
ticular Unguistic example in Sec. 12.51 Next, we build an ICS/DCM architecture in 
Sec. [3]by mapping filler/role decompositions of minimalist data structures onto ten- 
sor product representations in geometric spaces. The main results of the section are 
summarized in two theorems about minimalist representation theory. We also intro- 
duce harmonic minimalist grammar (HMG) here, by proposing a harmony metric for 
minimalist trees in representation space. In Sec.|4]we resume the linguistic example 
from Sec. l2.5l and construct arithmetic and fractal tensor product representations for 
our minimalist toy-grammar The paper concludes with a discussion in Sec.|5] 



2 Minimalist Grammars Revisited 



In this section we rephrase derivational minimalism ('StabieTlQQT',' St abler and KeenanI 
[2003, Michaelis 2001) in terms of term algebras (Kracht 2003) for feature strings 
and trees which is an important prerequisite for t he aim of t his stu dy, namely vector 
space representation theory. Moreover, following Harkemal ( 2001 ). we disregard the 
original distinction between "strong" and "weak" minimalist features that allow for 
"overt" vs. "covert" movement and for merge with or without head adjunction, re- 
spectively. For the sake of simplicity we adopt the notations of "strict minimalism" 
(Stabler 1999, Michaelis 2004), yet not taking its more restricted move operation, the 
specifier island condition (iGartner and Michaelisll2007h . into account. 
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2.1 Feature strings 



Consider a finite set of features Fp and its Kleene closure Fp. The elements of Fp 
can be regarded as terms over the signature Ff U {e}, where the empty word £ has 
arity and features f E Fp are unary function symbols. Then, the term algebra Tp is 
inductively defined through (1) e e T/r is a term. (2) If i g ?> is a term and / e Fp, 
then f{s) e Tp. Thus, a string (or likewise, an array) s = /1/2 . . .fp G Fp, p E No, of 
features /,■ e Fp is regarded as a term s = (/i 0/2 o . . .ofp){e) = /i (/2(. . . (/p(e)))) G 
7/7, where "o" denotes functional composition (the set No contains the non-negative 
integers 0, 1,2, . . .). 

First, we define two string functions for preparing the subsequent introduction of 
minimalist grammars. 



Definition 1 Let s ETphea feature string with s = f{r), f E Fp, r E Tp. 



1. Iht first feature of s is obtained by the function first : Tp \ {e} — >■ Fp, first(,s') = 
first(/(r)) =/. 

2. In analogy to the left-shift in symbolic dynamics dUndand Marcudll995h we de- 
fine shift : Tp \ {e} -> 7>, shift(5) = shift(/(r)) = r. 



Basically, the functions first and shift correspond to the LISP functions car and cdr, 
respectively. 



2.2 Labeled trees 



In early MG, a minimalist expression is a finite, binary, and ordered tree endowed 
with the relation of (immediate) projectio n among siblings and with a la beling func- 
tion mapping leaves onto feature strings ('Stablei) ll997 , Michaelis 2001 ). Such trees 
become terms from a suitably constructed term algebra 7a, as follows. As signature 
of Ta we choose the ranked alphabet A = 7> U {<, >}, where Tp is the previously in- 
troduced algebra of feature strings, and rank^ : A No. Feature strings are ranked as 
constants through rank^ {s) ~ for all s E Tp. Furthermore, the minimalist projection 
indicators, <, >, are regarded as binary function symbols: rankA(<) = rankA(>) = 2. 
Then we define by means of induction: (1) Every sETpisa. term, seT^. (2) For terms 
tQ,t\ E Ta, <(fo,fi) E Ta and >(fo,fi) E Ta. Then, <(fo,fi) denotes a minimalist tree 
with root <, left subtree fo and right subtree t\. The root label < indicates that fo 
"projects over" f]. By contrast, in the tree >(fo,fi) fi "projects over" Jq. 



Definition 2 A minimaUst tree f e 7^ is called complex if there are terms fo,?! E Ta 
and f — > or f — < such that t = /(fo, f 1 ). A tree that is not complex is called simple. 
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In correspondence to [Smolensky and Legendre (l2006ah and ISmolenskvl (l2006h . 



we define the following functions for handling minimalist trees. 

Definition 3 Let f e 7a be given as t — /(fo,fi) with f — > or f — <, to,ti e 7^. 
Then we define 

1. Left subtree extraction: exo :Ta -^Ta, 

exo(f) =tQ. 

2. Right subtree extraction: exi :Ta ^T^, 

exi(f) =fi . 

3. Tree constructions: cons/- : Ta x Ta ^ Ta, 

cons/(fo,fi) = f . 

Recursion with left and right tree extraction is applied as follows: 

Definition 4 Let / = {0, 1}* be the set of binary sequences, Y — YiYi ■ ■ -Yn & t, for 
n G No- Then the function exy : 7^ — > 7^ is given as the concatenation product 

eXf = id 

where id : 74 — > 7)v denotes the identity function, id(f ) = f, for all f e Ta- The bit 
strings 7 G 7 are called node addresses for minimalist trees and 7 is the address space. 



Using node addresses we fetch the function symbols of terms through another 
function. 



Definition 5 Let f e 7^ be given as f = /(fo,fi) with f — > or f — <, fo,fi G Ta, and 
Ye I. Then label -.IxTa^A with 

label(e,f) = / 

label(i7,f) = label(7,ex,(f)) . 
If f is a constant in Ta, however (i.e. f G 7>), then 



for every 7 G 7. 



label(7,f) = f , 
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Corollary 1 As a collorary of definitions\3\and\5\we state 

f =consiabei(£,o(exo(f):exi(f)) , (1) 
j/rankA(label(e,f )) ^2fort e Ta. 

Definition 6 The head of a minimalist tree f € 7a is a unique leaf that projects over 
all other nodes of the tree. We find f's head address by recursively following the 
projection labels. Therefore, head : 7^ — > / is defined through 

head(< (fo,fi)) = 0'"head(fo) 
head(> (fo,fi)) = l"^head(fi), 

where string concatention is indicated by '"^"^ and 

head(f ) = e , 

for / e Tf 

Definition 7 The feature of a tree t is defined as the first feature of f's head label. 
Thus feat :Tj\ Ff, 

feat(f) = first (label (head (f),f)) , 

where we appropriately extended domain and codomain of first : A\{e} ^ FpU {e}, 
by setting first(<) = first(>) = e. 

A node in a minimalist tree t is known to be a maximal projection if it is either 
f's root, or if its sister projects over that node. We exploit this property in order to 
recursively determine the address of a maximal subtree for a given node address. 



Definition 8 Let f e T4 and ye/. Then, max : I x ^ I, 



max(7,f) = i'~"max(5,ex,(f)) 
undefined 



7= head(f) 

7 = i8 and 7 ^ head(f ) 

otherwise 



is a partial function. 



We also need a variant thereof with wider scope. Thus we additionally define: 



Definition 9 Let f C / be a set of node addresses, then max* : p{I) xTa^ P{I), 

max*(P,f) = y {max(7,f)} . 

If f is a singleton set, = {7}, we identify the actions of max and max*. Here, p{I) 
denotes the power set of node addresses I. 
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Moreover, we define a function that returns the leaf addresses of a tree t possess- 
ing the same feature f ^ Fp. 

Definition 10 Let t e 7a and f eFf. Then, leaves ■.Ff'xTa^ P(7), with 

leaves(/,f) = {ye /|first(label(7,f)) = /} . 
where 7 varies over the address space of t. 

Next, we introduce a term replacement function. 

Definition 11 Let f , f' e 7^ and ye/. Then replace : / x I4 x 7^ — > I4 with 

replace(e,f,f') = t' 
replace(07,f,f') = consiabei(£,,)(replace(7,exo(f),f'):exi(f)) 
replace(l7,f,f') consiabei(£,,) (exo(f),replace(7,exi (f ),f')) ■ 

Using replace we extend the domain of the shift function ([U from the string algebra 
Tp to the tree algebra Ta ■ 

Definition 12 Let f e 7a. Then, shift* -.Ta^Ta with 

shift* (f) =:replace(head(f),f,shift(label(head(f),f))) , 
deletes the first feature of f's head. 



The effect of the tree functions head and max are illustrated in Fig.[T] The head 
of the tree f is obtained by following the projection indicators recursively through the 
tree: head(f) = 0~^head(exo(f )) = 00'"head(exo(exo((f)))) = 001'"head(exi (exo(exo((f ))))) = 
001; and max(100,f) = l'^max(00,exi(f)) = 1. 




Fig. 1 Labeled minimalist tree t with leaf addresses for illustration of head and max functions: 001 
head(i) and, e.g., max(100,f) = 1. 
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2.3 Minimalist grammars 

Now we are prepared to define minimalist grammars in term algebraic terms. 

Definition 13 A minimalist grammar (MG) is a four-tuple G = (P,C,Lex,./#) obey- 
ing conditions (1) - (4). 

1. f is a finite set of non-syntactic phonetic features. 

2. C — B U S U LU M is a finite set of syntactic features, called categories, compris- 
ing basic categories, B, selectors, S, licensors, L, and licensees, M. There is one 
distinguished element, c G B, called complementizer. Ff = P U C is then the fea- 
ture set. To each selector s € S a basic category b E B is assigned by means of a 
select function, sel: S ^ B, b — sel(i). Likewise, a license function, lie : L — > M 
assigns to each licensor £ E La corresponding licensee through m = lic{£). 

3. Lex C Tf is a finite set of simple terms over the term algebra 7>, called the lexi- 
con, such that each term t S Lex, is a feature string of the form 



4. ^ = {merge, move} is a collection of partial functions, merge : and 
move -.Ta^Ta, defined as follows: The domain of merge is given by all pairs of 
trees Dommerge = {(f 1,^2) € 7a X 7A|sel(feat(fi)) = feat(f2)}. The domain of move 
contains all trees Dommove = G 7A|feat(f) G L and max*(leaves(lic(feat(f )),?),?) 
contains exactly one element}. Let fi ,f2 € Dommerge and f g Dommove, then 



The constraint on the move operation, that the set of maximal subtrees with the 
corresponding licensee may contain exactly one element is called the shortest move 
condition, motivated by linguistic considerations. Relaxing this condition yields dif- 
ferent kin ds of minim alist grammars that could account for particular locality condi- 
tions (Gartner and Michaelis 2007) . 



2.4 Processing algorithm 



S*{L{J{e))S*BM*P* . 



merge(fi,f2) = 



move(f ) 



J cons<(shift*(ri),shift*(f2)) if t\ is simple 
\ cons>(shift*(fi),shift*(f2)) if t\ is complex 

COnS> (shift* (eXmax(leaves(lic(feat(f )) ,f) (0 ) i 

shift* (replace (max (leaves (lie (feat (f ) ) , f ) , f ) , f , e ) ) ) ) 



Minimalist grammar recognition and parsing are well understood dHarkemal 12001 , 
[Mainguv 2010, Stabler 2011). However, for our current exposition, instead of a fuU- 
fletched minimalist parser that must be proven to be sound and complete, we discuss 
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a simplified processor for our particular example from Sec. l2.5l below. just in order to 
provide a proof-of-conc ept for our representation theory. To this end we utilize early 
ideas of Stabler (Il996h as employed by Gerth (2006) and Gerth and beim Graben] 



(l2009h . There, the structure building functions merge and move are extended to a 



state description, or a stack, regarded as a finite word of terms w S T^. From a graph 
theoretical point of view, a state description is an unconnected collection of trees, 
and therefore ?i forest. In order to construct an algorithm that generates a successful 
derivation we introduce the following extensions of merge and move over forests of 
minimalist trees. 



Definition 14 Let w — [wi , W2, . . . , Wm) G 7^ be state description. Then 



1. merge* -.T^-^T^ with merge* (w) — {wi,W2,- ■ . ,merge(w,„_i , Wm)), when {w„^i,\ 

Dommerge ■ 

2. move* :T^-^T^ with move*(w) — (wi, W2, . . . ,move(wm)), when w„, e Dommove- 



are partial functions acting upon state descriptions from T^. 

In Def. [141 merge* operates on the next-to-last and the last element of the pro- 
cessor's state description, respectively, thereby implementing a stack with the last 
element at the top. Using this convention, canonical subject-verb-object sentences, 
[S[V011, such as the example below and also examples used by Gerth (2006 ) and 



Gerth and beim GrabenI (l2009b . can be processed straightforwardly, by first merg- 
ing the verb V with the direct object O as its complement, and subsequently merg- 
ing the result with the subject noun phrase. Thereby, the procedure avoids unnec- 
essary garden-path interpretations. However, since minimaUst languages cannot be 
processed with simple pushdown automata, one needs additional mechanisms such 
as indices and sorting in the framework of multiple context-free languages for which 
the crucial soundness a nd completen ess properties of minimalist parsers have been 
proven ('Maing uvl2010l. lStabler 201 iS. In our simplified approach, however, we make 
use of an oracle for rearranging stack content. This is implemented through suitable 
permutations tt : 7^ — > 7^, acting upon the stack according to w' = n{w). 

The processor operates in several loops: two for the domain of merge and another 
one for the domain of move. In the loops for the domain of merge the iteration starts 
with the tree on top of the stack which is checked against every other tree whether 
they can be merged, in which case an appropriate permutation brings both trees into 
the last and next-to-last position of the stack. Then merge* is applied and this loop 
iteration is terminated. If the top tree cannot be merged then the algorithm decrements 
backwards until it reaches the first tree on the stack. In the loop for the domain of 
move every tree is checked for being in the domain of move, in this case the move* 
operation is used after a permutation bringing that tree into the last position of the 
stack. The rest of the lexical entries in the state description are passed on unchanged 
to the next state of the algorithm. 
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Therefore after merge* or move* has been applied to the state description the 
algorithm completes the current state and continues with the next one resulting in a 
sequence of state descriptions So,Si,... which describes the derivation process. The 
algorithm stops when no further merge* or move* is applicable and only one tree 
remains in the state description. This final state description determines the successful 
derivation. 



2.5 Application 

We illustrate the procedure from Def. [T4]by constructing a minimalist grammar for 
the following English sentence and by outlining a successful derivation of 

(1) Douglas loved deadlines^ 

The minimalist lexicon is shown in Fig.|2] The first item is a complementizer (basic 
category c) which selects tense (indicated by the feature — t). The second item is 
a determiner phrase "Douglas" (basic category d) requiring case (licensee —case). 
The third item, the verb "love" (category v), selecting a determiner (feature = d), is a 
verb (feature v) and is moved into the position before "-ed" triggered by — i resulting 
in the inflection of the verb (i.e., "loved"). The next item would normally include 
an affix (e.g., -ven, -ing) but it is empty (e) here, it selects a verb (feature — v), a 
determiner phrase (feature = d) to which it assigns case (feature +CASE) and has the 
feature v. The fifth item represents the past tense inflection "-ed" with the category t 
that selects a verb (= v), assigns case (licensor +CASE) to a determiner and contains 
the licensor +1 to trigger the movement of "love". The last item in the lexicon is the 
object "deadlines" (category d) which requires case (—case). 



d 

—case 
deadlines 



Fig. 2 Minimalist lexicon of sentence |(l)| 



The algorithm takes initially as input the state description wi = (Douglas, love, 
-ed, deadlines) G T^. 

' Douglas Adams was quoted as saying: "I love deadlines. I like the whooshing sound they make as 
they fly by." in Simpson, M. J. (2003). Hitchhiker: A Biography of Douglas Adams. Justin, Charles and 
Co., Boston (MA). 



d 

— case 
Douglas 



"=d" 




— V 

+CASE 




— V 

+1 


V 

— i 




= d 




+CASE 


love 




V 

. e . 




t 

. -ed . 
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2.5.1 An example derivation of sentence \( 1 )\ 



Starting with the initial state description wi the words "love" and "deadlines" are 
merged (Fig.O after a first permutation ni, exchanging "-ed" and "love", by applying 
merge* (7i:i(wi)) =(Douglas, -ed, merge(love, deadlines)) because "love" (— d) and 
"deadlines" (d) are in Dommerge- 




love 



Fig. 3 Step 1 : merge. 



In the next step e is merged to the tree. 




— i 




—case 


love 




deadlines 



Fig. 4 Step 2; merge. 



The resulting tree is in the domain of move triggered by the features —case and 
+CASE, therefore "deadlines" is moved upwards in the tree leaving behind X, a new 
leaf node without label. The involved expressions are co-indexed with k (Fig.|5]l. 



deadlines 




Fig. 5 Step 3: move. 
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In Step 4 the whole state description W2 = (Douglas, -ed,fi) is checked for being 
in the domain of merge. This is the case for (Douglas, fi). Therefore, "Douglas" is 
merged to t\ . 




Next, the past tense inflection "-ed" is merged to the tree triggered by v. 




Now, the tree is in the domain of move triggered by — i and +1. Therefore, the 
maximal projection love A/; undergoes remnant movement to the specifier position in 
Fig.l 
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The resulting tree is again in Donimove and "Douglas" is moved upwards leaving 
a X behind indexed with j (Fig.|9]l. 




In the final step, the complementizer "c" is merged to the tree leading to the final 
minimalist tree with the unchecked feature c as its head (Fig.fTOli that completes the 
successful derivation. 
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3 Integrated Symbolic/Connectionist Architectures 



Connectionist models of symbolic computations are an import ant branch in cogni- 
tive s cience. In order to construe compositional representations jpodor and Pvlvshvn 
19881) one has to solve the famous binding problem known from the neurosciences 



dEngel et a Jl997h : ' low are representations from different perceptual modalities bound 
together in the representation of a complex concept? The same problem appears 
for complex da ta structures such as lists or trees, e.g., in computational linguistics 
(lHagoort||2005l) : Aow is a syntactic category bound to its functional role in a phrase 
structure tree? 



A solution for this binding problem has b een provided by Smolensky's Integrated 
Conne ctionist/Symbolic Architectures (ICS) ( Smolenskvl200a. Smolenskv and Legendre 
|2006a''b^. Here, complex symbolic data structures are decomposed into content fillers 
and functional roles that bind together in a geometric representation by means of ten- 
sor products. A closely related approach is Dynamic Cognitive Modeling (DCM) 

(Ibeim Graben and Potthastll20Q9t |in press), where neural network models are explic; 

itly co nstructed from geometric representations by solving inverse problems dPotthast and beim GrabenI 
2009h . 



In this section, we apply the concepts of ICS/DCM to our reconstruction of min- 
imahst grammars and processor, obtained in Sec.|2l 
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3.1 Filler/role bindings 



Consider a set of symbolic structures S and some structure s G S. A filler/role binding 
of s is then a set of ordered pairs j3 (s) of fillers bound to roles. 



Definition 15 Let F be a finite set of simple fillers and /? be a finite, countable, or 
even measurable set of roles. By induction we define a family of complex fillers as 
follows: 

Fo^F 

Fr,+ l = p{Fn X R) , 

where « e No and p{X) denotes the power set of some setX. Furthermore we define 
the collection 

F„=RU (^[jF,}j . 
The filler/role binding for 5 is a mapping j3 : 5 — > /v,. 



In the simplest case, simple fillers are bound to roles. Thus, a filler/role binding 
P i^) ~ {(/' '") 1/ G ■f'l G ^} G P{F X 7?) = Fi. Such a decomposition could act as a 
complex filler /' for another filler/role binding where /' = j3 (s) is bound to a role r, 
resulting in j3(i') = {{/' ,r)\f' E Fi,r E R} G p{F\ xR) = F2. By means of recursion 
any fi nite structure of arb itrary complexity yields its filler/role binding as an element 
of Foo dbeim Graben et al2008b,) . 



Next we construct filler/role bindings for minimalist trees, 5 = 7^, in a hierarchi- 
cal manner. To this aim we start with feature strings. 



3.1.1 Feature strings 



Let S — Tfhe the string term algebra over signature Fp U {e} from Sec. 12. II A string 
* = (/i °/2 ° • • • ° fp){£) € Tf assumes a straightforward filler/role binding by inter- 
preting Fp as the filler set. Then each string position / is identified with one role. 
Si e Rp, such that Rf = {si\i £ N} is an infinite but countable set of roles. However, 
since every string s ETp is of finite length p, only roles from Rp = {si\l < i < p} are 
required. 



Definition 16 An order-reverting filler/role binding jip : Tp ^ P{Ff x Rp) for fea- 
ture string s — f{r) G Tp of length p > is given as a mapping 

j3F(e) - 
^H/W) = {(./,^p)}Uj3fW- 



16 beim Graben & Gerth 

As an example consider the term (/i o /2) (e) e 7>. Its filler/role binding is then 

/3F(/l(/2(e))) = {(/l,^2)}U/3f(/2(e)) = {(/l,.2)}U{(/2,.l)}Uj3f (£) = {(/!,. 2), (/2,^l 

3.1.2 Labeled trees 



The fill er/role binding for labeled binary t r ees ha s been discussed bv lbeim Graben et al 



(l2008allbh . and lbeim Graben and Potthastl ( l2009l) . ' "or tree term algebras 7^ from Sec. 12. 21 



we identify the signature A = 7>U{<,>} with the set of simple fillers and introduce 
roles Ra = {?"o,'"i,'"2}, with "mother" (r2), "left daughter" (ro) and "right daughter" 
(ri) of an elementary tree as indicated in Fig.[TT] where the indices have been chosen 
in accordance with the extraction functions exo and exi from Def.[3] such that exo(f) 
is bound to role ro and exi(f) is bound to role r\ for a term f S J4. In accordance 
to Def. [15] we call the set of complex fillers Aoo. Additionally, we unify the sets of 
simple fillers and roles through 

F = /=>U{<,>} (2) 
R=RfURa. (3) 



Fig. 11 Elementary roles of a labeled binary tree. 



Definition 17 A filler/role binding Pa'.Ta^ Aoo for tree terms is given as a mapping 

Consider the minimalist tree t — >{f,g) G 7^ in Fig. fT2l where the root is labeled 
with the projection indicator pointing to the head at the right daughter and feature 
string terms / = (/i o /2 o . . . o fp){e) e Tf, p £ g = {gi o . . . o g^){e) e Tf, 
q GN, are presented as column arrays. 





> 








'gi' 


fi 




82 


jp. 







Fig. 12 Minimalist tree term t = >{f,g) e with feature gi. 
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The filler/role binding of t is obtained as 

j3^(r)=j34>(/,g))-{(>,r2),(/3A(/),ro),(/3A(g),n)}-{(>,r2),(i3f(/),ro),(i3Hg),'-l)}^ 

= {(>,'-2),({(/l,ip),(/2,Sp-l),---,(/p,^l)},'-o),({(gl,i«),(^2,ig-l),---,(g9,il)},'-l)}- 

A more complex expression s = >{f,<{g,h)) G 7^ is shown in Fig.[T3] 





< 




'gl' 






gl 




hi 






A. 



Fig. 13 Complex minimalist tree i = >(f ,<(g,h)) G 7^ with feature gi . 



The filler/role binding for the term s in Fig.[T3]is recursively constructed through 

(.)= J3a (>(/,<(§, /i) )) = {(>, r2) , (/3a (/), ro) , (J3a (< fe, ), )} = 
= {(>,r2),(j3f(/),ro),({(<,r2),(j3A(^),ro),(j3A(/i),ri)},ri)}- 
= {(>,r2),(i3f(/),ro),({(<,r2),(/3f(g),ro),(j3f(/i),n)},n)} = 
= {(>,r2),({(/i,ip),(/2,Sp-i),...,(/p,si)},ro), 
{{{<,r2),{{{g\,Sq),{g2-,Sg-i), . . . ,{gc,.,si)},rQ), 

{{{hi,Sr),{h2,Sr-l),. . . ,{hr,si)},ri)},ri)} . 



3.2 Tensor product representations 



Definition 18 Let ^ be a vector space over the real or complex numbers, and j3 : 
S Foo a filler/role binding for a set of symbolic structures S for sets of fillers F and 
roles R. A mapping i// : /v, =^ is called tensor product representation of S if it 
obeys (1)- (3). 



1. y/{F„) is a subspace of JF, for all n e No, in particular for Fq = F is \lf{F) = "fp 
a subspace of ^ , 

2. \l/{R) = % is a subspace of ^, 

3. r)}) = v^(/) ^ y/{r), for filler / G F„ and role r e R (n e No). 

4. v^(A UB) = i/(A) ® for subsets A,B C /v,. 
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Lemma 1 ^ is the Fock space 

^^[^^^F®rf^®rR, (4) 

known from quantum field theory \Smolenskv and LesendrS2006A) . 



Proof (by induction over n G No). Let n = 0. Then 'fp (g) —Yf is a subspace of 
^. Moreover Yris a subspace of J^. Let f & F„, r e R, such that <//(/) e wiFn) and 
{(/, r)} e Fn+i be a filler/role binding. Then r({(/,r)}) = «) r(r) € r(F„+i). 
The direct sum of those subspaces is the Fock space 

By concatenating the maps /3,V^, we extend the tensor product representation 
^ . S ^ ^ of the symboUc structures: 

n^) = r(/3(5)), seS. (5) 



Definition 19 Let j3 , be a filler/role binding and a tensor product representation 
for a structure set S in Fock space ^ over fillers F and roles R. A linear function 
Vr : ^ is called unbinding for role r if 



(xifif) : u = V/({(/,r)}) 
1^ : otherwise. 



Unbinding functions can be esta bhshed in several ways, e.g. by rneans of adjoint vec - 
tors or through self-addressing ("S molenskv and L egendre'l 2006a , Smolensky 2006 ). 
Self-addressing requires that the Fock space ^ is equipped with a scalar product, 
turning it into a Hilbert space. However, in this paper, we use adjoint vectors, i.e. 
linear forms into its number field, from the dual space of the respective role 
representation space. This requires that all filler and role vectors are linearly inde- 
pendent, implying faithf ul tensor product representations dSmolenskv and Legendre 
2006ai ISmolenskvll2006b . 



Next, we define the realization of a symbolic computation. 



Definition 20 Let P,Q : S ^ She partial functions on the symbolic structures s G S, 
such that Codp C Domg. Two piecewise Unear functions P,Q : ^ ^ are called 
realizations of the symbolic computations P,Q in Fock space jF, if there is a tensor 
product representation f : 5 — > such that 

(PoW){s) = (f oP)(s) 

(Qo'f^)(f) = (f oe)(f) 



for all s e Dom/), f G Domg. 
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Then, the realizations constitute a semigroup homomorphism and hence a semigroup 



representation in the sense of algebraic representation theory (Ivan der Waerde 
Ibeim Graben and Potthas t 2009), because 



rug 



2003 



(QoPof)(,9) = (f oeoP)(s) 



for all s G Domp. 



3.2.1 Feature strings 



In Sec. 13.1.11 we created filler/role bindings for the term algebra Tp of minimalist 
feature strings as j5f{s) ^ I3{{fi 0/20 .. .o fp){e)) = {{fi,Sp),{f2,Sp^i), ... ,{f,,,si)} 
in reversed order by regarding the features as fillers Fp and the string positions Rp = 
{si\i e N} as roles. Mapping all fillers /,■ e Ff onto filler vectors f,- = wifi) ^ 
from a vector space ff, and similarly all roles Sj e Rp for a string of length p onto 
role vectors s,- = e from a vector space Yr yields a tensor product 

representation of feature strings in preserved order through 

f(,v) = VA(j3f(i)) = f f;®s,-. (6) 

1=1 

However, for the sake of convenience, we extend the representation space to an em- 
bedding space spanned by the role vectors that are required for representing the 
longest feature strings. Let therefore « e N be the maximal length of a feature string 
occuring in the minimalist lexicon, we bind the null vector to all role vectors Si^ for 
k > p for a given string of length p <n. Then, all strings possess a unique represen- 
tation 

f(.) = V/(j3f(*)) = £f;®s,-. (7) 

1=1 

We denote the embedding space for feature strings ,5^. 

For this representation we have to find realizations of the string functions from 
Def. [U To this end we need some preparatory concepts. Let u ~ f (5) = \I/{I5f{s)) 
be a tensor product representation for feature strings s — (/i o /2 o . . . o /p) (e) e Tp, 
fi G F. For the role vectors s, G Vr we define their adjoints G Yr in the dual space 

of Yr, such that 

si{sk) = 5,k, (8) 

with the Kronecker symbol 5,i = 0(1) for / ^ k (i ^ k), i.e. the adjoint vectors s^^, 
acting as linear forms, and the duals s, form a biorthogonal basis. 



Lemma 2 X)k{\i) = (id® s^)(u) is an unbinding function for role s^.. 
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Proof. Let u = ik®H- Then 

Vk{vL) = (id®s^)(u) = {\d®sl){tk®Sk) = h4i^k) = ^Ak Vifk) ■ 

Here, id : ^ 1^ denotes the identity map at fp- id(f) = f. 

Since is also linear, we additionally obtain the following result: Let u = Y!i=i fi €5 
Sj. Then 

Vk{u) = (id(^s+)(u) = (id®s+) ^£f/«)S, j = 

= t (id® s+)(f,- ^ s,) = £ f,-s+(s;) = f f,-5,,- = f, = v/(/^) . 

Definition 21 Let 'f' be a tensor product representation of terms of feature strings Tp 
in vector space y, and u — f (i) ~ ^4=1 ® S; for f £F*. 

1. The first feature of u is obtained by an unbinding function first \ ~^ ^ with 

first(u) = (id(g)sf)(u) (9) 

2. A function sliift : — !• is obtained by 

(1-1 

sliift(u) = £((id«)s/^i)(u))(g)S,- + 0«)S„ (10) 

1=1 



Lemma 3 first and sliift are realizations of the corresponding string functions first 
and shift /rom Def. |7] 



/'roo/. Let s = /(r) e 7>. Then 

first('f^(i)) = f = v/(/) = f (first(i)) . 
For sliift(u) we compute 

sliift('f^(5)) = ^(id(X)S+ 1) {^h^^^j^ (8)S,+0(g)S„ = 

M— 1 / 71 \ 



i=\ \k=\ ) 

(I-l 

£ £ (8) s; + (g) s„ = £ f,-+i (g) s, + (g) s„ = f (shift(s)) . 



i=lir=l (=1 
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3.2.2 Labeled trees 

A tensor product representation of a labeled binary tree is obtained from the respec- 
tive filler/role binding in Def . [Tt] 

Deflnition 22 Let t — /{toji) e 2a be a tree term with f — > or f — <, to,ti e Ta. 
Then 

^{t) = r(/3A(0) = r({(/,r2),(j3A(fo),ro),(j3A(fi),n)}) = f®r2®'f^(fo)0ro©f (fi)®ri , 

where the projection indicators, < and > are mapped onto corresponding filler vec- 
tors f< — y/{<), f> = W{>)^ ^nd the three tree roles "mother", "left daughter", and 
"right daughter" are represented by three role vectors Tq = i/A(ro),ri ~ w{''i)j^2 = 
Wi^i) G Moreover, we also consider their adjoints r([,r]^,rj e from the dual 
space ^* for the required unbinding operations. 

Using Def. |22] together with the unified sets of fillers and roles from Eqs. (O 
we can compute tensor product representations of minimalist trees, as those from the 
examples of Sec. 12.21 The tensor product representation of the tree term t = >{f,g) E 
Ta in Fig. [12] is given as 



f (0 = r(i3A(>(/,^))) - r({(>,r2),(j3A(/),ro),(j3Ate),n)}) = 

= r({(>,'-2),(i3F(/),'-o),(j3F(g),n)})-f><^r2®l/({(/l,5p),(/2,ip-l),...,(/p,5l)})(^ro® 

= f> (gi r2 ® (f 1 (g) Si e f2 (8i S2 ® • • • ® fp «) Sp) (Xi ro ® (gi (Xi si ® g2 (Xi S2 ® • • • ® «) s,) «) r 1 
which can be simplified using tensor algebra to 

f (0 =f> (8)r2®fi (Xsi iX)ro®f2(8)S2(8)ro® • • •®fp(8)Sp(8)ro®gi (Xsi (Xri ®g2X)S2X)ri ® • • •®gg(8)S^(8)ri . 

(11) 

Correspondingly, we obtain for the tree term s = >{f,<{g,h)) e Ta depicted in 
Fig. [T3]the tensor product representation 

f (i) = f > (g) r2 ® f 1 (X s 1 (g) ro ® f 2 (X S2 (X ro ® • • • ® fp (g) Sp (g) ro® 
®f<g)r2g)ri®gig)Sig)rog)ri®g2g)S2g)rog)ri® •••®g^g)S^g)rog)ri® 
® hi g) si g) ri g) ri ® h2 g) S2 g) ri g) ri ® • • • ® hr g) Sr g) ri g) ri . 

Interestingly, leaf addresses 7 = 7i 72 • • • 7p G /, /5 G N, correspond to role multi-indices 
by means of the following convention 

Ty ry, g) g) • • • g) Ty, . (12) 



22 



beim Graben & Gerth 



Using role multi-indices Ty, we can introduce further generalized unbinding functions 
below. 

Now we are prepared to define the Fock space realizations of the tree functions 
from Sec. 12] Before we start with the counterparts from Def.[3j we notice an interest- 
ing observation. 

Lemma 4 Let Ta be the term algebra of minimalist trees and W its tensor product 
representation in Fock space as above. For u € the function first distinguishes 
between simple and complex trees. 

1. u = f (f) with t £Tf is a simple tree (i.e. a feature string). Then 

first (u) 7^0. 

2. u = ^{t) with t = f{to,ti) (^Ta is a complex tree. Then 

first (u) =0. 



Proof. Consider the first case: u = 'f'(f) for simple t, hence u = f; (Xis,. Then 
first(u) = fi 7^ 0. For the second case, we have u = ^{t) —t®r2® f (fo) ® fo © 
'f^(fi)«)ri. Therefore 

first(u) = (id(g)s+)(u) = (id(8)S+)(f«)r2©f (fo)«)ro®f (fi)«)ri) = 

fs+(r2)®'f^(fo)s|(ro)®f(fi)s+(ri)=0, 

where id denotes the Fock space identity appUed to the respective subspacesH 



Definition 23 Let Ta be the term algebra of minimaUst trees and f its tensor product 
representation in Fock space as above. Moreover, let u € ^ with first(u) = 0, i.e. 
the tensor product representation of a complex tree. Finally, let uo,Ui G ^. 



1 . Left subtree extraction: exo : ^ — > 

exo(u) = (id(g)rj)(u). 

2. Right subtree extraction: exi : ^ — ^ 

exi(u) = (id(8)rf )(u) . 



Note that the Fock space identity id can be expressed as a direct sum of the respective sub- 
space identities id = Y.k^'^k- Applying that to an arbitrary Fock space vector u = 12x"A yields id(u) = 
Y.kY.1 idK(u;^) = u, such that id|c(u;t) = for K=/^X. Hence, the identities for different subspaces behave 
like orthogonal projectors, annihilating vectors trom their orthocomplements. This observation applies also 
below. 
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3. Tree constructions: cons/ : ^ x ^ — > 

cons/(uo,ui) ==uo®ro©ui(g)ri®f/®r2. 

Lemma 5 exo, exi and cons/ are realizations of the corresponding string functions 
exo, exi and cons f from Def.\3\ 

Proof. Suppose t = f{to,t\),tQ,t\ e I4. Then 

exo(f (f)) = (id^r+)(f (/(fo,fi))) = {iA®r+){i®r2®w{h{to))®rQ®w{^A{h))®ri) = 
fr+(r2)® r(j3A(fo))r+(ro)® v^(j3A(fi))r+(ri) = wiMo)) = f (fo) - f (exo(O) • 

The proof for exi works similarly. Furthermore, 

cons/(f (fo),'f^(fi)) = f(fo)«)ro®'f^(fi)(8)ri®f/(8)r2 f (cons/(fo,fi)) . 

Next, we extend those functions to node addresses as in Def. |4] 

Definition 24 Let / = {0, 1 }* be the set of binary sequences, 7 = 71 72 ■ • • 7n G /, for 
n G Nq. Then the function exy : ^ ^ is given as the concatenation product 

exg = id 

ex,y — ex; o exy . 

Then, we get the following corollary from Lemma |5j 

Corollary 2 Functions exy are realizations of the corresponding string functions 
from Def 

Next, we realize the symbolic label function in Fock space. 

Definition 25 Let Ta be the term algebra of minimalist trees and f its tensor product 
representation in Fock space as above. Moreover, let u G ^ and y € I. Then 
label : / X ^ ^- ^ with 

label(£,u) = (id(g)rj)(u) 
label(/7,u) = label ( 7, ex,- (u)) , 

when first(u) — 0. If first(u) 7^ 0, then 

label(7,u) = u. 
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Lemma 6 label is a Fock space realization of the term algebra function label from 
Def.m 

Proof. First assume first (u) 7^ 0, then u G ^ is the tensor product representation of 
a string term t ETp labeHng a leaf node in a tree. Therefore 

label(7,'f^(f)) = 'Fit) = f (label(7,f)) • 

Next, suppose first(u) ~ 0, in which case u G ^ is the tensor product representation 
of a proper tree term t — f{to,ti) ET/^.By means of induction over 7, we have first 

label(e, f (f )) = (id (g) r+) (f (g) ® f (fo) <^ ro © f (fi) n ) = f = 'F(label(e,0) 

and second 

label(/7,f (0) =label(7,ex,(f(f))) =label(7,f (ex,(f))) = "f'(label(7,ex,(f))) = f (label(!7,f)) . 



The following function does not provide a Fock space realization but rather a kind 
of Fock space isometry. 



Definition 26 The head of the tensor product representation of a minimalist tree f G 
Ta is obtained by a function head : I, 

( e iffirst(u)7^0 

head(u) ~ < 0^head(exo(u)) if u = cons<(exo(u),exi(u)) 
[ l'^head(exi(u)) if u = cons>(exo(u),exi(u)) . 

Lemma 7 Let t G Ta- Then 

head('f^(f)) =head(f) 

with head given in Def. |6] 

Proof (by induction). First assume that first(u) 7^ 0, i.e. t E Tp with u = 'f'(f) is a 
head. Then 

head(f (f)) = e = head(f). 

Next suppose t ~ /(?(),?! ) G Ta with either / = < or / = > . In the first case we have 
u = cons<(exo(u),exi (u)) and therefore 

head('F(f)) = 0'^head(exo(u)) 0'^head(exo(f )) , 

in the second case we haveu = cons>(exo(u),exi(u)) and thus 

head('F(f)) = l'^head(exi(u)) = l'^head(exi(f)) . 
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Definition 27 The feature of the tensor product representation u = ^{t) of a mini- 
malist tree t is retained as the first feature of f 's head label. Thus feat : J? Yf, 

feat(u) = first(label(tiead(u),u)) . 
Lemma 8 Let t € Ta- Then 

feat(>f'(0) = f (feat(f )) 

with feat given in Def. 

Proof. Follows immediately from previous lemmata and definitions. 

Also the maximal projection becomes an analogue to a Fock space isometry. 

Definition 28 Let u e =^ and ye/. Then, max -.Ix.^^I, 

{£ 7 = liead(u) 

;'^max(5,ex,(u)) : 7= /5 and 7^ liead(u) 
undefined : otherwise . 

Lemma 9 Let t € Ta and y &L Then 

max(7,f (f)) =max(7,f) 

with max given in Def. |^ 

Proof (by induction over 7). Let 7 = liead(f(f )). Then 

max(7,'f^(f)) = e = max(7,f) , 
if 7 7^ tiead('f^(f )), by contrast, we find 5 G / such that 7 = /5, hence 

max(7,f (f)) = /^max(5,ex;('f^(f))) = i""max(5,exi(f)) =max(7,f). 

The function max is naturally extended to sets of node addresses. 

Definition 29 Let u e ^ and PcL Then, max* : p{I) x ^ ^ p{I), 
max*(f,u) = y {max(7,u)} . 
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Next we have to adapt the definition of the symboUc leaves function from Def.fTOl 
The corresponding realization leaves : X/r x ^ ^ p{I) is obtained from a generalized 
unbinding function 

ubfeat(7,f,u) = (f+(g)s+(g)r+)(u) , (13) 

for given filler vector f E Yp and leaf address 7 G /, applied to the tensor product 
representation of a tree t £ T^, because all first features of the tree's leaves built 
partial sums of the form 

m 

^f,'(8)Si®r;,, , (14) 

1=1 

as they are bound to the first role Si in the feature lists. Here, r,j, denote the multiple 
tensor products of roles according to Def. [121 

Applying Eq. ( fT3] l to this expression yields 

Cm \ m 

f; ® si ® r,,, 1 = L (f'- )Ki^i)4i^v.)^h^ 

for all f, = f. 

Therefore we get 

Definition 30 Let f e 1^ and u e ^ . Then, leaves : x ^ -> 
leaves(f,u) = {ye /|ubfeat(7,f,u) = 1} . 

Lemma 10 Let t e Ta and f e Ff. Then 

leaves('f^(/), f (f)) = leaves(/,f) . 

Proof. The lemma follows from the above calculation. 
Next, we modify the replacement function. 

Definition 31 Let u, u' e and ye/. Then replace -.Ix-^xJ^^.^ with 
replace(e,u,u') = u' 

replace(07,u,u') = consiabei(f,u)(replace(7,exo(u),u'),exi (u)) 
replace(l7,u,u') = consiabei(£,u)(exo(u),replace(7,exi(u),u')) ■ 

Lemma 11 Let t,t' £Ta and y £l. Then 

replace(7,'f^(f),'f^(f')) = f (replace (7,?, f')) 
with replace /rom Def.UT] 
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Proof (by means of induction over 7). First let 7=6. Then 

replace(e,'f'(f), f (f')) = 1'{t) = f (replace(£,f ,?')) . 

Next assume that Lemma [TT] has akeady been proven for all address strings 7 of 
length p e Nq. Then iy with / = or / = 1 is of length p-\-\ and it holds either 

replace(07,f(f),f(f')) = cons,abei(£,tf'(r))(replace(7,exo(f(f)),f(f')),exi(f (?))) = 
cons^(iabei(e,,))('i^(replace(7,exo(f),f')),f (exi(f))) = 'f^(consiab<.i(£_,)(replace(7,f,f'),exi 

f (replace(7,f,f')) 

or 

replace(l7,f(f),f(f')) = consiabei(.,nO)(exo('i'(0),replace(7,exi('f^(f)),f (?'))) = 
cons^(iabei(£,r))('f^(exo(f)),'i'(replace(7,exi(f),f'))) = 
'F(consiabei(£,,)(exo(f),replace(7,exi(f),f'))) = f (replace(7,f ,f')) . 

Using the Fock space realization of replace we also extend the domain of the shift 
function (I2TI) from string vectors in 5^ to tree vectors in ^ . 

Definition 32 Let u G ^. Then, shift* : ,^ ^ with 

shift* (u) = replace(head(u),u,shift(label(head(u),u))) . 

Lemma 12 Let t e Ta- Then 

shift*('f^(f)) = 'f' (shift* (f)) 

with shift* /rom Def. [72] 

Proof. The Lemma follows from previous observations. 



3.3 Minimalist grammars 

In this section we introduce geometric minimalist structure-building functions and 
prove that they are indeed Fock space realizations of the term algebraic functions 
from Sec. 12.31 
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Definition 33 Let G = {P, C, Lex, ^) be a minimalist grammar (MG) with phonetic 
features P, categories C = BUSULUM, lexicon Lex C Ta, and structure-building 
functions ^ = {merge, move} as defined in Def. [13] Let sel : 5 B be the select 
function and lie : L — > M be the license function. Moreover, let 'f' = y/o be a tensor 
product representation of the term algebra 7^ of G on Fock space We introduce 
realizations sel: ^ ^ and lie : ^ -> ^ by demanding 

f (sel(s)) = sel(f 
f (lic(£)) = lie(f (^)) 

for 5 e 5 and £ Q L. The domain of merge is given by all pairs of vectors Dommerge = 
{(ui,U2) € =^ X ^|sel(feat(ui)) = feat(u2)}. The domain of move contains all vec- 
tors Dommove = {u e ^|feat(u) e 'f^(L) and max*(leaves(lic(feat(u)),u),u) contains 
exactly one element}. Let ui ,U2 G Dommerge and u G Dommove, then 

f cons<(sliift*(ui),sliift*(u2)) if first (m) 7^ 
\ cons>(sliift*(ui),sliift*(u2)) if first(ui) = 

C0nS>(sllift*(eXi„ax(leaves(lic(feat(u)),u),u)(u))7 

sliift* (replace (max (leaves (lie (feat (u )), u) , u) , u , e )) ) 



merge (ui,U2) ^ 
move(u) ~ 



Theorem 1 Let be the minimalist tree term algebra and f its tensor product 
representation in Fock space as above. Let f 1 , f2 G Dommerge <^nd t G Domn,ove> 
then 

merge('F(fi),f (f2) = f (merge(fi,f2)) 
move(f (0) = f (move(f)) 

with merge, move /rom Def.\T3\ 



Proof. The Theorem follows from the Lemmata in Sec. 13.21 



Taken together, we have proven that derivational minimalism ( Stabled 1997 , Stabler and Keenan 
20031 lMichae"Iis ''200l') can be realized by tensor product representations as a starting 



point for integrated con nectionist/symbolic architectures (.Smolensky and Legendre 



2006aL ISmolenskvll2006h . 



3.4 Processing Algorithm 



In order to realize a minimalist bottom-up processor as discussed in Sec. l2.4l in Fock 
space, we have to represent the processor's state descriptions (Stabler 1996). This can 
be achieved through another filler/role binding by introducing new roles pi,p2, - ■ ■ G 
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R for stack positions binding minimalist trees. Then the tensor product representation 
of a state description w of length m assumes the form 

m 

w=^Wi:«)p^, (15) 

k=l 

where w^^ are tensor product representations of minimalist trees. 

The minimalist algorithm as defined in Def. [14] becomes then realized by corre- 
sponding Fock space functions merge* and move*. 

Definition 34 Let be the minimalist tree term algebra and ^ its tensor product 
representation in Fock space as above. Furthermore, let ^ be augmented by the 
role vectors of a minimalist state description. We define 

1 . merge* : ^ ^ with 

m— 2 

merge*(w)= ^ (id«)p+)(w)«)pi,emerge((id(g)p+_i)(w), (id«)p+)(w))«)pm_i . 

k=\ 

2. move* : ^ ^ ^ with 

m-l 

move*(w) = (id®p^)(w)(8)p^©move((id(8)p+)(w))(g)pm. 

k=\ 

In Def.[34]the adjoint vectors p^ applied to the tensor product representation w yield 
the corresponding expressions w^, from Eq. ( fTsT l. Clearly, this definition entails a min- 
imalist processor as stated by the next theorem. 

Tlieorem 2 Let Tf^ be the set of minimalist expressions and f the tensor product 
representation of its state descriptions in Fock space as above. The functions 
merge* and move* given in Def. realize a minimalist bottom-up processor in 
Fock space. 

The proof of Theorem |2] requires the realizability of permutation operators 77 : 
^ — >^ ^ in Fock space. Such general permutations can be assembled from elemen- 
tary transpositions T,-y, exchanging items / and j in an ;«-tuple. The corresponding 
realization T,y is then obtained in the following way. Let 

m 

W = ^ Wi: (8) Pi: 

k=l 

be the state description in Fock space and P,y be the projector on the orthocomplement 
spanned by p, and py . Then 

T,;(w) =P,7(w) + (id(g)p+)(w)(8py + (id(g)p+)(w)®p,- (16) 

realizes the transposition T,y in Fock space ^ by means of unbinding functions. Then 
entries in the state description can be the rearrangement such that merge* and move* 
as defined in Def . |34]become applicable. 
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3.5 Harmonic minimalist grammars 



A crucial component of ICS is harmony theory. At the symbolic level of description, 
harmony assesses the well-formedness of a structure by means of soft-constraints re- 
warding the minimization of markedness. It can be gauged in such a way, that totally 
well-formed output assumes harmony // = 0. By contrast, at the subsymbolic level 
of description, harmony provides a Lyapunov function guiding the computational dy- 
namics by means of gradient ascent. In a neural network realization harmony of an 
activation vector v is given by a quadratic form 

//(v) =v+-W(v)-v, 

where v+ denotes the transposed of v and W(v) is the synaptic weight matrix in 



state V corresponding to the com putational function applied to v (iSmolenskv l2006[ 
Smolensky and Legendr 



We owe a fir st indication of weighted or harmonic minimalist grammars (HMG) 
to lStablen(ll997h who speculated abou t "additional 'eco n omy princip les,' acting as a 
kind of filter on derivations" (see also ^ Harkema toih v lHaigtoeh made the first 
attempt to implement this idea by constructing probabilistic context-free grammars 
from minimalist derivation trees. Therefore we suggest the following definition. 



Definition 35 A harmonic minimalist grammar (HMG) is a minimalist grammar G 
(Def. [T3] l augmented with: 



1. A weight function for feature terms W :Tf ^ 0p=i such that W{s) is a p- 
tuple {xi,X2,. ■ ■ ,Xp) eM.'' of real weights assigned to a feature term s = (/i 0/2 o 
. . . o /p) (e) of length /? G N. In particular, W assigns weights to the features in the 
minimalist lexicon Lex. 

2. A harmony function for trees // : 7^ — > M, given by 

H{t) = x+(H'(label(head(f),f))) , 

with the adjoint vector of the direction of the xpaxis: {xi,X2, ■ ■ ■ ,Xp) = xi, 
returns the weight of f's head. 

3. A collection of partial functions hmerge : R x Ta x Ta ^ x Ta and hmove : 
R X 7a — >■ R X I4, defined as follows: 

hmerge(/!,fi,f2) = (/i + //(fi) +//(f2),merge(fi ,f2)) 

hmOVe(/z,f) (/i + //(f) +^^(eXmax(leaves(lic(feat(f)),f),f)(f)):mOVe(f)) , 

for heR. 

4. The harmony filter. A minimalist tree t £ Ta is harmonically well-formed if it is 
MG well-formed and additionally 

h{t)>0, 

where h{t) is the cumulative harmony of t after application of hmerge and hmove 
during the derivation of f , starting with initial condition ho E M. 
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Next, we suggest a metric for geometric representations that can the regarded as 
a measure of harmony. For that aim we assume that the Fock space ^ is equipped 
with a norm 1 1 • 1 1 : ^ — > R^j" assigning a length | |u| | to vector u G ^ . Such a norm 
could be supplied by a scalar product, when ^ is a Hilbert space. 

Definition 36 Let (wi,W2, . . • ,wr), W|t G 1 < ^ < T e Nbe a (finite) trajectory 
in Fock space of duration T, representing a minimalist derivation with initial state wi 
and final state wr. We define harmony through the distance of an intermediate step 
wj^ from the well-formed parse goal wj-, i.e. 

Hiy^k) = -||wi-wr|| . 

Lemma 13 The harmony function from Def. |56] is non-positive for all processing 
steps and increases towards H = when approaching the final state, H{wt) = 0. 

Proof. The Lemma follows immediately from Def. [36l 

Eventually we combine Def. [35] and Def. [36] by looking at harmony differences 
AHii = //(w<;+i) — H{wii) between successive parse steps. These differences can be 
distributed among the features triggering the transition from Wjt to Wjt+i, as will be 
demonstrated in Sec. 14.31 HMG could then possibly account for gradience effects in 
language processing. 



4 Applications 

In this section we present two example applications which use the tensor product rep- 
resentations of Sec. l3.2l in different ways. Both representations are given here, since it 
is the aim of this paper to give theoretical justifications for both at the same time. The 
representations are using tw o different en codings. At first we show arithmetic rep- 
resentations implemented by Gerth ('2006'), then, we describe fractal representations 
outlined by Gerth and b eim Graben (.2009.) . For computing harmony we use Euclid- 
ian norm in both cases. 



4. 1 Arithmetic Representation 

In a first step, we map the fillers F for the features of the lexical items onto 12 filler 
vectors as shown in Tab. [1] 

In order to ensure a faithful representation, filler vectors need to be linearly inde- 
pendent, i.e., they form a basis of 12-dimensional vector space. Trying to implement 
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d 


fl 


= d 


h 


V 




= V 


U 


t 


ts 


= t 


fe 


+CASE 


fv 


— case 


fs 


+1 


fs 


— i 


fio 


> 


fll 


< 


fl2 



Table 1 Fillers for the minimalist lexicon outlined in Fig.|2] 



this requirement, leads to an explosion of dimensions (more than 5 millions) which 
was beyond the limits of memory on the used workstation. Therefore, we refrained 
from linear independence and used a linearly dependent, distributed, representation 
of filler vectors in a 4-dimensional vector space f, e M^, (1 < ; < 12) instead. 

The actual filler vectors are: 



11 = 







(') 




(') 









,f2 = 


1 


,f3 = 





,u = 











1 




















1 

V3 



1 
1 

1 
1 



,f6 



1 

73 



1 
1 

V 1 J 
1 

V 1 J 



1 

V3 



fii = ^ 



1 

V3 



/ 1 \ 

-1 

1 

V 1 J 

( ' \ 

1 
1 

V 1 J 



1 

V3 



/ 1 \ 

1 

-1 

V 1 J 
1 

-1 



Similarly, the tree roles from Fig. [TT] are represented by three-dimensiona l basis 
vectors as achieved in previous work (beim Graben et al 2008a, Gerth and bei m GrabenI 
2009h . Further, we need to map the list positions Si {\ <i < 4) of the features onto 
role vectors. Therefore, a total of 3 + 4 = 7 role vectors is required. Again we have 
to use a linearly dependent representation for role vectors because of an explosion of 
dimensions and a restriction on available workstation memory. 

In particular, we make the following assignment for tree roles "left-daughter" 
To = ei; "right-daughter" rj — 62; "mother" r2 = 63, where e^t (k = 1, 2, 3) are the 
canonical basis vectors of three-dimensional space R^. The roles of list positions in 
the feature arrays of the minimalist lexicon rt^i = s, (1 < i < 4) are indicated in 
Fig.lHl 
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d rj 
— case r4 
Douglas 



= d rs 

-i rs 
love 



= V rs 
fCASE r4 
= d rs 
V rs 

e 



= V rs 
+1 r4 
f CASE rs 
t rs 
-ed 



d rj 

—case r4 
deadlines 



Fig. 14 Roles for the Minimalist lexicon outlined in Fig.|2] 



The vectors for the hst positions are distributed on the unit sphere in ! 





1 Lre: 




The following example shows a tensor product representation of the lexical item 
for "love": 



"=d 


fi 




"=d 


ra 


V 






V 




— i 


fio 





— i 




love 






love 





^ f2 81 r3©f3(g)r4© floors 



Fig. 15 Tensor product representation of the lexical item "love". 
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In our arithmetic tensor produc t representation, tensor products are then given as 
Kronecker products (IMizraiill 19921) of filler and role vectors, f, ® r^., by: 





(ro\ 


fl 


® . 


\fj 





fl 



fl 



fn 



V6/ 

fro\ 



//i'-o\ 
/in 

f\ri 

fm 

fxH 
fir 5 
fire 
firo 
fir I 
firi 
firi 
fir A 
firs 
fire 

fiiro 
fliri 
fliri 
fliri 
flirA 
furs 
KfiireJ 



In order to construct an appropriate embedding space, we chose the largest tree 
appearing in the minimalist state description. The tensor product representation of 
every tree t E Ta is then embedded into that space by left-multiphcation of the tree- 
roles with sufficient tensor powers 



)r2( 



(p times) of the mother role, where the exponent p e No is different for every tree. 



Finally, we have to construct the tensor product representation for the state de- 
scriptions of a minimalist bottom-up processor as described in Sec. 13.41 Here, we bind 
all minimalist expressions to only one role pQ for the state description. For the ten- 
sor product representation, we simply choose po = 1, i.e. the scalar unit. As a result, 
all tree representing vectors become linearly superimposed in the state description 
dSmolenskv and Legendrell2006al) . 
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4.2 Fractal Tensor Product Representation 



Gerth and beim Graben ([2009') introduced a different encoding called fractal ten- 



sor product representation which is a combinatio n of the arithmetic description in 
the previous section and scal ar Godel encodings (Ibeim Graben and Potthast l2009[ 



Gerth and beim Grab en 2009). For a fractal representation we encode the three tree 



roles rQ,ri,r2 localistically by the canonical basis vectors of three-dimensional vec- 
tor space as above. However, fillers for minimalist features are represented by integer 
numbers g{fi) from a Godel encoding. The Godel codes used in our example are 
shown in Tab. |2] 



Filler /, 


Code 


d 





= d 


1 


V 


2 


= V 


3 


t 


4 


= t 


5 


+CASE 


6 


— case 


7 


+1 


8 


— i 


9 


> 


10 


< 


11 



Table 2 Fractal encoding for minimalist lexicon in Fig.|2] 



The role vectors of the tree positions are mapped onto three-dimensional vectors 
in the same way as described in Sec. 14.11 The only difference is the encoding of the 
positions of the lexical items in the feature array. Here, the roles si^ are encoded by 
fractional powers A'^* of the total number of fillers, which is = 12 and k denotes 
the k-th list position. The following example shows the lexical entry for "love" and 
its fillers represented as Godel numbers: 



-'love 



= d 1 

V 2 
-i 9 
love 



It becomes described by the sum of (tensor) products of Godel numbers for the 
fillers and fractions for the list positions: 

giLu„.e) = 1 X 12"' +2 X 12-^ + 9 X 12^^ = 0.1024. 



The next example illustrates the encoding of a subtree, consider the tree: 
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Its encoding is given through 

g{<)(g>r2®g{L,)ro®g{Lr)(E)ri 

11 X 12"' I j +(2x 12"'+9x 12-^) 







• 


+ 7 X 12"' 








(17) 



where L; and Lr denote the feature arrays of the left and right leaf. Complex trees are 
again represented by Kronecker products (see Sec. l4.1l for details). 

The state description of the algorithm is mapped step by step onto the fractal 
tensor product representation. At first, each leaf in the tree is encoded in an enumer- 
ation of fractals. In the second step the encoding of the whole state description is 
achieved by recursively binding minimalist trees as complex fillers to 3-dimensional 
role vectors. Finally the representation of all trees in the state description is linearly 
superimposed in a suitable embedding space. 



4.3 Results 



In this section we present the results of the applications obtained in the previous 
sections (Sec.|0] Sec.l4!2b. 

The final derivation of the minimalist algorithm (Sec. 12.51 ) results in a matrix 
which is the state space trajectory. Each column stands for one derivational step in 
form of a vector in a high-dimensional embedding space. The dimensions of the final 
embedding space are li = 78732 for the arithmetic representation and d = 6561 for 
the fractal tensor product representation. 

For visualization purposes the data have to be compressed. A common technique 
in multivariate statistics is the principal component analysis (PCA), which ha s been 
used as an observable model previously (beim Graben et al 2008a, Gerth and beim Grabenl 
2009h . Before applying the PCA the trajectories are standardized using z-transformation 
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to obtain a transformed distribution with zero mean and unit variance. Then the great- 
est variance in the data is in the direction of the first principal component, the second 
greatest variance is in the direction of the second principal component and so on. 
Plotting the first, PC#1, and the second, PC#2, principal component as observables 
against each other, entails a two-dimensional phase portrait as an appropriate visual- 
ization of the processing geometry. 

First, we present the phase portrait and the harmony time series from Def. [36lof 
the arithmetic representation for sentence [(l)| in Sec. l2.5l in Fig.[T6l 

Figure [T6l a) shows the phase portrait in principal component space. Each parse 
step is subsequently numbered. Figure [T6l b) presents the temporal development of 
the harmony function. 

The derivation unfolds as described in Sec. 12.51 The initial state description (step 
1) represents the lexicon and starts in coordinate (—1.72,-1.43) in Fig.fTSl'a) with a 
harmony value ofH — —6.49 [Fig.[T6l'b)1. As the parse continues the harmony trajec- 
tory climbs steadily upwards. In parse step 3 e is merged to the tree [Fig. fTSl a)]: co- 
ordinate (-3.83,-7.3)). Interestingly the graph of the harmony reaches a local min- 
imum inH — —6.08 here and continues again upwards until parse step 8 [Fig. [T6l a)l: 
coordinate (-5.05, 15.70); Fig.fTelb): H = -4.76. In this step the subject "Douglas" 
is moved upwards leading to the final phonetic, but not yet fully syntactically parsed, 
representation of the sentence. In the end the graphs reach their final states in coordi- 
nate (-2.51, -0.7 l)[Fig.[l6l;a)] and in H ^ [Fig.\W[b)]. 




(a) 



(b) 



Fig. 16 Results for the arithmetic representation (Sec. l4.lt . (a) Phase portrait of the first principal com- 
ponent, PC#1, versus the second principal component, PC#2. (b) Harmony time series from Def. l36l 



Figure[T7]shows the observables for the processing mapped onto the fractal repre- 
sentation. Figure [TTl a) displays the phase portrait in principal component space. Be- 
sides the apparent nonlinearity, one realizes another interesting property of the fractal 
representation: While the minimalist processing unfolds, the feature arrays contract. 
This is reflected by the increasing phase space volume available to the geometric 
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dynamics. As above, Fig. fTTl b) illustrates the temporal development of the harmony 
function. Again, the initial state description represents all entries in the lexicon which 
starts in coordinate (—0.03,0.07) in Fig. fTTl a) with a harmony value of H = —2.63 
in Fig.[T7fb). In comparison to Fig. fTSl a) the representations of the first seven parse 
steps stay close to each other before deviating to coordinate (—0.25,6.09) in step 8. 
The harmony curve in figure Fig. fTTl' b) exhibits a downwards trend. Like in Fig. [T6l b) 
the graph of the harmony reaches a local minimum in parse step 3 (// = — 3.4) when 
£ is merged to the tree [Fig.[T7l'a)1: coordinate (-7.3,-5,8). Finally the end states 
are reached in coordinate (6.8,-0.94) [Fig. fTTl a)] and in a harmony value of H — 
[Fig.inib)]. 




(a) (b) 

Fig. 17 Results for the fractal representation (Sec. l4.2t . (a) Phase portrait of the first principal component, 
PC#1, versus the second principal component, PC#2. (b) Harmony time series from Def. l36l 



Table [3] summarizes the evolution of harmonies for both representations. 



Representation 


Step: 1 


2 


3 


4 


5 


6 


7 8 9 


arithmetic (Sec. 14. It 


-6.49 


-5.96 


-6.08 


-5.3 


-5.36 


-5.08 


-4.94 -4.76 


fractal (Sec.l42l 


-2.63 


-2.97 


-3.4 


-3.33 


-3.6 


-3.8 


-3.74 -3.71 



Table 3 Harmony time series for both tensor product representations. 



Finally, we construct HMGs from these data by assigning harmony differences 
to the features of the minimalist lexicon as follows: First, we compute harmony dif- 
ferences 4/4 = //(wa+i) — //(wa) between successive processing steps from Tab.[3] 
Then, the difference 4/4 is assigned to either a selector or a licensor that triggers the 
transition from Wjt to w^+i while the corresponding basic categories or licensees are 
weighted with zero. 

Figure [TS] depicts the resulting HMG lexicon for the arithmetic representation 
(Secgjli. 
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Fig. 18 Harmonic minimalist lexicon of sentence|(l)|obtained from arithmetic representation (Sec. 14. 11 



Moreover, Fig.[T9]shows the HMG lexicon for the fractal representation (Sec. l4.2l l. 
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Fig. 19 Harmonic minimalist lexicon of sentence|(l)|obtained from fractal representation fSec. l4.2t . 



5 Discussion 



In this paper we developed a geometric representation theory for minimahst gram- 
mars (MG). We resumed minimalist grammars in terms of partial functions acting 
on term algebras of trees and feature arrays. Those complex data structures were 
mapped onto vectors in a geoni etric space (known as the Fock space ( Haad 1992 , 
Smolenskv and Legendre'2006a*)') using filler/role bindings and tensor product repre- 



sentations ( Smolenskv and Legendre 2006a, Smolensky 2006, beim Graben and Pot thasi 
2009h . We were able to prove that the minimalist structure-building functions merge 



and move can be realized as piecewise linear maps upon geometric vector spaces. In 
order to present a proof-of-concept, we generalized the merge and move functions to- 
wards state descriptions of a simple derivation procedure for minimahst trees which 
also found a suitable realization in representation space. In addition, we suggested 
a harmony function measuring the distance of an intermediate processing state from 
a well-formed final state in representation space that gave rise to an extension of 
MG towards harmonic MG (HMG). This harmony observable could be regarded as a 
metric for processing complexity. While our proofs essentially relied on faithful rep- 
resentations, we used two different kinds of non-faithful, distributed representations 
in our numerical applications. Firstly, we employed arithmetic vector space encod- 
ings of minimalist features, roles and trees. Secondly, we used fractal tensor product 
representations that combine arithmetic vector spaces with numeric Godel encod- 
ings. For both cases, we presented phase portraits in principal component space and 
harmony time series of the resulting minimalist derivations. Finally, we derived the 
corresponding HMGs from simulated harmony differences. 
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Our theory proves that sophisticated grammar formalisms such as MG can be re- 
ahzed in a geometric representation. This would be a first step for dynamic cognitive 
modeling of an integrated connectionist/symbolic (ICS/DCM) architecture for pro- 
cessing minimalist grammars. Since natural languages t entatively belong to the sam e 



complexity class of mildly context-sensitive languages jShiebe: 



; ly belong to th e same 
3l985llStable3l2004 



ICS/DCM architectures are principally able to process natural language. However, 
the simple processing algorithm used in the present study just for ill ustrating the rep- 



resentation theory, is not a so und and complete minimalist parser (iHarkemal 12001 



[Mainguy 2 O10l IStabler 1201 11) . Therefore, future work towards psycholinguistically 
more plausible processing models, would comprise the development of a geomet- 
ric representation theory for chain-base d minimalism and for multiple context-free 
parsing (Harkema 2001, Stabler and Ke enan 2003). 

Moreover, processing minimalist grammars by ICS/DCM architectures straight- 
forwardly provides a notion of harmony. However, a proper treatment of HMG would 
require further investigations to be carried out: Our definition of harmony in Def. [36] 
combines a particular metric (e.g. Euclidian) with one well-formed reference state 
Wr for minimalist processing, while harmony in ICS is defined as a general quadratic 
form only depending on the synaptic weight matrix. Therefore, one has to examine 
how these expressions would transform into each other. Moreover, HMG lexicons in 
the sense of Def. [35] could also be trained from large text corpora, e.g., in order to 
explain gradience effects. Then one has to check how subsymbolic harmony would 
be related to soft-constraint harmony obtained from corpus studies. 



The requirements of our theory for tensor product constructions to be faithful 
representations of minimalist processing lead to extremely high-dimensional embed- 
ding spaces. These spaces contain extremely few symbolically meaningful states. 
Therefore, numerical application on common workstations is only feasible by us- 
ing compressed and thus non-faithful representations. Yet, non-faithful representa- 
tions are also interesting for more principal reasons, as they allow for memory ca- 
pacity constraints, e.g. by means of graceful saturation in neural network models 
( Smolensky and Legendre 2006a , [Smolensky 2006) . Several possible compression 
techniques have been suggested in the Uterature, e.g. contraction (i.e. outtrac eing). 



circular convolu tion, holographic reduc ed representations, o r geometric algebra (Co ecke et all 
201 lilAerts et al 2009. Plate 200 3. Smolensky and Legendre 2 006a, Smolensky 20(3^ 
beim Graben and Potthast 20091) 7 It would therefore be necessary to generalize our 
current theory to compressed representations, including an assessment of the entailed 
representation errors. We leave this issue for future work. 



Another important aspect of our work concerns the relationship between mini- 
malist grammar and compositional semantics. On the one hand, it is straightforward 
to include semantic feature s into minimalist lexicons, e.g. as type-logical expressions 
(INivogi and Berwick I2OO5 I). On the other hand, this is somewhat redundant because 
the v ery same information is already encoded in the minimalist features (iKobele 
20061) . Vector space semantics a ppears as a very powerful tool fo r combining corpus- 
driven latent semantic analysis (ICederberg and Widdowsll2003b with compositional 
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semantics based on com pressed tensor product representations ( Blutner 20091 Aerts 
ICoecke et alll201 In our geometric representation theory, syntactic roles and 



thereby also semantic functions are encoded by node addresses in high-dimensional 
tensor products of role vectors for tree positions. Therefore, one should seek for ap- 
propriate unbinding maps that could be combined with their semantic counterparts 
dCoecke et a il EoT Ij). Also this promising enterprise is left for future work. 
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